COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking

This paper presents COOL-MC, a tool that integrates state-of-the-art reinforcement learning (RL) and model checking. Specifically, the builds upon OpenAI gym probabilistic checker Storm. COOL-MC provides following features: (1) simulator to train RL policies in for Markov decision processes (MDPs) are defined as input Storm, (2) new builder which uses callback functions verify (neural network) policies, (3) formal abstractions relate models specified or (4) algorithms obtain bounds on performance of so-called permissive policies. We describe components architecture demonstrate its features multiple benchmark environments.